Enable tuning in Batch norm CK solver #3326
Conversation
bghimireamd
requested review from
BrianHarrisonAMD,
junliume,
randyspauldingamd and
iq136boy
and removed request for
junliume and
JehandadKhan
|
IsApplicable/GetSolution in general should throw instead of asserting for several reasons:
assert(i < vec.size());
auto x = vec[i];With |
| @@ -240,7 +240,8 @@ MhaCKFlashAttentionV2Forward::GetSolution([[maybe_unused]] const ExecutionContex | |||
|
|
|||
| fmha_runtime_args.p_drop = probability; | |||
| fmha_runtime_args.drop_seed_offset = | |||
| std::make_pair(dataFwd.dropoutSeedData, dataFwd.dropoutOffsetData); | |||
| std::make_pair(reinterpret_cast<uint64_t>(dataFwd.dropoutSeedData), | |||
There was a problem hiding this comment.
@CAHEK7 had to this to silence the compiler.
There was a problem hiding this comment.
I'd like to double check - are you going to convert gpu pointer to integer? Is the code using fmha_runtime_args.drop_seed_offset aware that it's actually gpu pointer?
For me that reinterpret_cast doesn't look like a correct solution.
There was a problem hiding this comment.
yes! its a GPU pointer but I think for now we have disabled the dropout https://github.com/ROCm/MIOpen/blob/develop/src/solver/mha/mha_ck_fa_v2_solver_forward.cpp#L234
There was a problem hiding this comment.
It would be better to make it explicitly 0 and put the same comment as
MIOpen/src/solver/mha/mha_ck_fa_v2_solver_forward.cpp
Lines 237 to 239 in 0c24160
|
@bghimireamd Windows builds are failing, the error messages look like something caused by this PR, could you take a look? |
This PR only enabling tuning in CK solver.